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BACKGROUND 


This  research  effort  supports  a  milestone  under  Science  and  Technology 
Objective  (STO)  E  (III. ME. 2004. 01),  Warfighter  Physiological  Status  Monitoring-Initial 
Capability  (WPSM-IC).  Commanders  and  medics  increasingly  require  remote,  real-time 
access  to  basic  physiological  information  about  their  Soldiers  in  order  to  make  better 
decisions  regarding  the  early  identification,  location,  and  triage  priority  of  casualties. 
Historically,  it  is  estimated  that  early  first  aid  intervention  may  save  up  to  20%  of  all 
battlefield  fatalities  (5,6).  A  life  sign  detection  system  (LSDS)  could  provide  combat 
care  providers  with  essential  vital  sign  information  that  would  assist  them  in  providing 
rapid  first  aid  to  battlefield  casualties.  Sophisticated  reliable  and  valid  output 
information  from  the  LSDS  could  lead  to  reduced  morbidity  and  mortality  of  both  medics 
and  Soldiers  by  facilitating  an  appropriate  medical  response.  Although  these  LSDS 
devices  represent  the  most  sophisticated  technology  currently  available  to  the  U.S. 
Army,  the  devices  have  never  been  tested  for  reliability  and  validity.  If  the  data 
provided  by  the  LSDS  is  reliable  and  valid  then  more  appropriate  decisions  may  be 
made  regarding  triage  priority.  Thus,  the  military  needs  to  identify  a  LSDS  device  that 
will  reliably  provide  the  combat  medic  with  valid  life  sign  information  about  the  Soldier 
under  operational  conditions. 
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EXECUTIVE  SUMMARY 


Life  sign  detection  system  (LSDS)  devices  were  developed  by  the  Center  for  the 
Integration  of  Medicine  and  Innovative  Technology  (CIMIT,  Cambridge,  MA,  USA), 
Hidalgo  Research  Co.  (Swavesey,  Cambridge,  UK),  Sarcos  Research  Co.  (Salt  Lake 
City,  UT,  USA),  and  VivoMetrics,  Inc.  (Ventura,  CA,  USA)  for  use  in  the  Future  Force 
Warrior  system.  This  study  evaluated  the  reliability  and  validity  of  these  four  LSDS 
devices  for  measuring  heart  rate  (HR)  and  respiration  rate  (RR)  against  criterion 
devices  (e.g.,  3-lead  ECG  and  metabolic  cart,  respectively).  Eight  male  volunteers 
(mean  ±  SD,  21  ±3  yr,  76±9  kg)  completed  four  days  of  testing  in  which  HR  and  RR 
were  collected  from  the  four  LSDS  devices  and  criterion  devices  at  the  same  time  for  ~ 
4  h  while  engaged  in  low,  medium,  and  high  intensity  activities  on  two  occasions.  The 
change  in  both  HR  and  RR  (i.e.,  error  scores)  between  the  two  trials  (trial  1 -trial  2)  and 
two  devices  (device-criterion)  was  calculated  and  analyzed  using  a  2-way  repeated 
measures  ANOVA.  Bland-Altman  plots  showed  the  dispersion  of  the  individual 
reliability  and  validity  error  scores.  All  devices  demonstrated  adequate  reliability  based 
on  hypothesis  testing  but  Bland-Altman  plots  revealed  tighter  prediction  intervals  for 
both  HR  (beatsvnin'1)  and  RR  (breaths*min_1),  respectively,  for  CIMIT  (±15.9  and  ±5.8) 
and  VivoMetrics  (±13.0  and  ±9.9)  compared  to  Sarcos  (±54.0  and±9.4)  and  Hidalgo 
(±86.6  and  ±14.2).  All  devices,  except  Sarcos,  demonstrated  adequate  validity  for  both 
HR  and  RR  based  on  hypothesis  testing  but  Bland-Altman  plots  revealed  tighter 
prediction  intervals  for  HR  (beatsTnin1)  and  RR  (breaths^min'1),  respectively,  for 
VivoMetrics  (±11.6  and  ±21.4)  compared  to  CIMIT  (±19.6  and  16.5)  and  Hidalgo  (±86.2 
and  15.7).  In  conclusion,  VivoMetrics  was  the  most  reliable  and  valid  LSDS  device  for 
measurements  of  HR  and  RR.  CIMIT  was  the  next  most  reliable  and  valid  device.  The 
Sarcos  and  Hidalgo  devices  should  not  be  evaluated  further  in  their  current 
configuration  based  on  the  poor  reliability  and  validity  of  HR  and  RR  measurements. 
Even  though  the  VivoMetrics  device  was  the  most  reliable  and  valid  device,  the  form 
factor  was  not  acceptable  to  the  Soldier  in  the  field.  Thus,  a  future  LSDS  system  may 
need  to  utilize  components  of  each  system  to  meet  all  the  needs  of  the  Soldier. 
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INTRODUCTION 


In  order  to  successfully  incorporate  any  piece  of  equipment  into  a  military 
system,  it  is  essential  to  determine  whether  the  piece  of  equipment  works  appropriately 
by  providing  reliable  and  valid  data  under  various  activity  levels.  Reliability  refers  to  the 
reproducibility  of  data  collected  from  equipment  in  repeated  tests  on  the  same 
individuals  (1,13).  Validity  refers  to  the  agreement  between  the  value  of  a 
measurement  and  its  true  value  (25,5).  The  concepts  of  reliability  and  validity  are 
related.  For  example,  measurements  can  be  reliable  but  not  valid,  but  a  valid 
measurement  must  be  reliable.  Several  different  methods  for  assessing  reliability  and 
validity  exist  (1).  The  five  most  common  measures  of  reliability  are  the  change  in  mean 
between  tests,  standard  error  of  measurement  (SEM),  coefficient  of  variation  (CV), 
which  is  the  SEM  expressed  as  a  percent  of  the  subject’s  mean  score,  Bland-Altman 
95%  limits  of  agreement,  and  intraclass  correlation  coefficient  (ICC).  The  five  most 
common  measures  of  validity  are  the  change  in  mean  between  devices,  the  estimation 
(i.e. ,  regression)  equation,  the  standard  error  of  the  estimate  (SEE),  Bland-Altman  95% 
limits  of  agreement,  and  the  Pearson  correlation  coefficient.  Since  arguments  for  and 
against  each  method  exist  depending  on  the  expert  quoted,  multiple  kinds  of  evidence 
should  be  presented  in  order  to  best  interpret  reliability  and  validity  data  (9,13). 

Reliability  and  validity  studies  are  routinely  done  on  new  pieces  of  equipment 
that  claim  to  provide  physiological  data  that  is  comparable  to  accepted  criterion 
methods  of  physiological  data  collection  (3,16-18,20,24).  Given  that  airway,  breathing, 
and  circulation  are  the  basic  ABC’s  of  maintaining  life  as  published  by  the  American 
Heart  Association  (10),  heart  rate  (HR)  and  respiration  rate  (RR)  are  critical 
physiological  measures  of  life  sign  status  in  the  Soldier.  Although  a  3-lead 
electrocardiograph  (ECG)  for  measuring  heart  rate  (HR)  and  a  metabolic  cart  for 
measuring  respiration  rate  (RR)  have  been  previously  validated  (15,21-23)  these 
criterion  methods  are  not  practical  for  operational  use  by  a  Soldier  in  the  field  because 
of  their  size,  weight,  and  inability  to  be  moved.  Therefore,  ambulatory  “wear  and  forget” 
life  sign  detection  system  (LSDS)  devices  were  developed  as  a  collaborative  effort 
between  the  U.S.  Army  Research  Institute  of  Environmental  Medicine  (USARIEM)  and 
the  following  four  companies:  (a)  Center  for  the  Integration  of  Medicine  and  Innovative 
Technology  (CIMIT,  Cambridge,  MA,  USA),  (b)  Hidalgo  Research  Co.  (Swavesey, 
Cambridge,  UK),  (c)  Sarcos  Research  Co.  (Salt  Lake  City,  UT,  USA)  and  (d) 
VivoMetrics,  Inc.  (Ventura,  CA,  USA).  The  LSDS  devices  provided  by  these  four 
companies  purport  to  provide  HR  and  RR  data  equivalent  to  previously-validated 
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criterion  methods.  However,  none  of  these  LSDS  devices  have  been  tested  for 
reliability  and  validity  on  human  subjects  under  a  variety  of  physical  activity  levels.  An 
independent  lab  validation  study,  therefore,  was  conducted  by  USARIEM  in  order  to 
more  clearly  identify  the  most  acceptable  LSDS  device  for  possible  insertion  into  the 
Future  Force  Warrior  (FFW)  military  system.  Furthermore,  since  engaging  in  military 
activities  is  likely  to  affect  the  signal  quality  of  recorded  physiological  data  (1 1 ), 
quantifying  the  percent  of  time  that  unusable  signals  are  generated  in  various  levels  of 
activity  (i.e. ,  low,  medium,  and  high)  will  enable  manufacturers  to  improve  and  enhance 
current  technologies  in  order  to  better  meet  the  Soldier’s  needs. 

OBJECTIVES 

The  first  objective  of  this  study  was  to  determine  the  reliability  and  validity  of  HR 
and  RR  collected  from  four  LSDS  devices  under  a  wide  range  of  low  (i.e.,  lying,  sitting, 
standing),  medium  (i.e.,  sit-ups,  push-ups,  jumping  jacks)  and  high  (i.e.,  walking, 
running)  intensity  activities  that  are  part  of  normal  military  duties.  Reliability  was 
determined  via  test-retest  methods  while  validity  was  determined  by  comparing  the  HR 
and  RR  data  collected  from  the  LSDS  devices  to  the  criterion  device.  The  second 
objective  of  this  study  was  to  determine  whether  any  of  the  LSDS  devices  provided 
acceptable  data  under  low,  medium,  and  high  activity  levels.  Acceptable  data  was 
defined  as  a  HR  and  RR  within  (a)  ±  3%  of  the  criterion  measure  90%  of  the  time  during 
low  activity,  (b)  ±4%  of  the  criterion  measure  80%  of  the  time  during  medium  activity, 
and  (c)  ±5%  of  the  criterion  measure  70%  of  the  time  during  high  activity. 

METHODS 


SUBJECTS 

Eight  male  soldiers  with  a  mean  (±SD)  age,  body  weight,  and  height  of  21±3  yr, 
76±9  kg,  175±5  cm,  respectively,  enrolled  in  this  study.  Each  gave  written  and  verbal 
acknowledgment  of  their  informed  consent  and  was  made  aware  of  their  right  to 
withdraw  without  prejudice  at  any  time.  Investigators  adhered  to  the  policies  for 
protection  of  human  subjects  as  prescribed  in  Army  Regulation  70-25,  and  the  research 
was  conducted  in  adherence  with  the  provisions  of  45  CFR  Part  46. 
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PROTOCOL 


Design 


Eight  Soldiers  wore  one  of  the  four  LSDS  devices  and  criterion  devices  at  the  same 
time  for  ~  4  h  each  day  while  engaging  in  low,  med  ium,  and  high  intensity  activities  on  two 
occasions  (Table  1 ). 


Table  1.  Schedule  of  Testing  Activities 


Activity 

Intensity 

Activity 

Time  of 

Activity 

Total  Time 

Low 

1 .  Lie  on  cot 

2.  Sit  on  cot 

3.  Stand  on  ground 

4.  Break 

5.  Repeat 

1 .  -10  min 

2.  -10  min 

3.  -10  min 

4.  -10  min 

5.  -40  min 

-  80  min 

Medium 

1.  Walk  on  treadmill  at  3.0  mph 

2.  Sit-ups,  push-ups,  jumping  jacks 

3.  Break 

4.  Repeat 

1.  -15  min 

2.  -10  min 

3.  -10  min 

4.  -35  min 

-  70  min 

High 

1.  Run  1 .0  mi  on  treadmill  at  6.0  mph 

2.  Break 

3.  Repeat 

1.  -10  min 

2.  -10  min 

3.  -20  min 

-  40  min 

The  HR  and  RR  were  collected  every  15-30  s  over  the  ~4  h  of  testing  from  both  the 
LSDS  and  criterion  device.  Each  day  of  testing  was  followed  by  at  least  one  day  of 
non-testing  and  no  exercise  such  that  volunteers  were  never  tested  more  than  three 
times  in  one  week.  Volunteers  were  tested  at  the  same  time  of  the  day  during  the  four 
days  of  testing.  The  study  design  was  counterbalanced  so  that  each  Soldier  wore  a 
different  LSDS  device  on  each  day  of  testing  to  control  for  order  effects  (Table  2). 
Soldiers  were  randomly  assigned  a  testing  order. 
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Table  2,  Counterbalanced  Experimental  Design 


Volunteer  /Device 

VivoMetrics 

Hidalgo 

Sarcos 

CIMIT 

1 

Day  1 

Day  4 

Day  2 

Day  3 

2 

Day  2 

Day  3 

Day  4 

Day  1 

3 

Day  3 

Day  2 

Day  1 

Day  4 

4 

Day  4 

Day  1 

Day  3 

Day  2 

5 

Day  1 

Day  4 

Day  2 

Day  3 

6 

Day  2 

Day  3 

Day  4 

Day  1 

7 

Day  3 

Day  2 

Day  1 

Day  4 

8 

Day  4 

Day  1 

Day  3 

Day  2 

Description  of  LSDS  Devices 

Vivometrics  LifeShirt™  is  an  FDA-approved  noninvasive  ambulatory  recording 
device  that  continuously  acquires  and  stores  respiration,  ECG,  and  body  position  data 
on  a  data  card  stored  within  a  portable  battery  powered  electronic  recorder  worn  on  the 
body.  The  LifeShirt  system  consists  of  a  LifeShirt  garment,  data  cable,  ECG 
electrodes,  recorder  battery,  data  cards,  electronic  recorder,  battery  charger  and 
recorder  case. 

Hidalgo  Thoracic  Sensor  is  a  prototype  chest-mounted  sensor,  worn  centrally 
on  the  thoracic  area  and  comprised  of  two  parts.  The  first  is  a  wireless  re-usable 
sensor  electronics  module  that  measures  ECG-based  heart  rate  and  respiration  rate. 
The  second  part  is  an  adhesive  electrode  disposable  laminate,  comprising  upper  and 
lower  sections,  both  of  which  have  a  removable  adhesive.  The  two  parts  are  joined  by 
metal  studs,  which  allow  the  upper  and  lower  parts  to  move  freely  with  respect  to  one 
another. 

Sarcos  Integrated  Sensor  Unit  is  a  prototype  physiologic  data  acquisition 
system  for  non-invasive  monitoring  of  vital  signs.  The  Integrated  Sensor  Unit  measures 
ECG-based  heart  rate,  presence/absence  of  breathing,  breathing  rate,  body  motion  and 
position,  and  axillary  temperature.  The  sensor  is  configured  in  a  form  of  a  chest  belt 
and  attached  to  the  chest  using  an  elastic  strap. 

CIMIT  Soft  Chest  Strap  is  a  prototype  device  intended  for  use  in  collecting 
physiologic  status  in  real  time.  The  components  of  the  CIMIT  Soft  Chest  Strap  include 
(a)  conductive  rubber  ECG  contacts  encased  in  a  polymeric  compound,  (b)  sensor  case 
made  of  a  rigid  polyurethane  material  which  is  designed  for  commercial  use  with  a 
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number  of  products  requiring  contact  with  human  skin,  and  (c)  the  polyester  and  Velcro 
chest  strap,  which  contains  no  latex. 

Description  of  Criterion  Devices 

The  Schiller  Cardiovit  AT-6  ECG  machine  (Schiller  Inc.,  Baar,  Switzerland)  was 
utilized  as  the  criterion  device  for  measuring  HR,  and  has  met  ECG  instrument 
specifications  of  the  American  Heart  Association  (2).  The  SensorMedics  2900 
automated  metabolic  cart  system  (SensorMedics  Corporation,  Yorba  Linda,  CA)  was 
utilized  as  the  criterion  device  for  measuring  RR,  and  has  been  previously  shown  to  be 
a  reliable  and  valid  instrument  for  measuring  respiration  (15,22). 

Environmental  Conditions 


All  testing  was  performed  in  the  Doriot  Climatic  chamber  located  in  Natick,  MA, 
at  the  Natick  Soldier  Systems  Center.  The  temperature  and  relative  humidity  in  the 
60x10x14  ft  climatic  chamber  was  maintained  at  21  ±2  °C  and  45±5  %,  respectively  for 
all  testing.  The  environmental  conditions  did  not  differ  on  any  of  the  four  testing  days. 

STATISTICS 

The  change  in  both  HR  and  RR  (i.e.,  error  scores)  between  the  two  trials 
(trial  1  — trial2)  and  devices  (device-criterion)  was  calculated.  Given  the  known  effects  of 
previous  activity  on  elevating  HR  and  RR,  a  negative  value  was  expected  for  the 
change  in  mean  between  trials  during  medium  and  high  activity.  However,  the  negative 
bias  should  be  similar  for  all  devices  given  that  the  activity  and  time  between  activities 
was  standardized  on  each  day  of  testing.  A  two-way  repeated  measures  ANOVA 
(device  x  activity)  was  utilized  to  determine  the  reliability  and  validity  of  each  LSDS 
device  during  each  activity  by  comparing  the  reliability  and  validity  error  scores  across 
devices  and  activities.  Individual  error  scores  of  zero  indicated  that  the  reliability 
(trial  1  —trial  2)  and  validity  (device-criterion)  of  the  device  was  high.  Bland-Aitman  plots 
were  constructed  to  show  the  dispersion  of  the  individual  reliability  error  scores 
(trial  1-trial2)  and  validity  error  scores  (device-criterion)  (7).  In  this  manner,  the  mean 
error  score  was  illustrated,  and  the  95%  limits  of  agreement  (prediction  interval)  were 
depicted.  Individual  error  scores  that  have  a  tight  prediction  interval  around  zero 
signified  a  more  reliable  and  valid  device.  In  addition,  the  coefficient  of  variation  (CV) 
and  intraclass  correlation  coefficient  (ICC)  were  calculated  for  both  HR  and  RR 
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measured  on  each  LSDS  device  and  criterion  method  during  each  of  the  eight  activities 
as  further  measurements  of  reliability.  The  standard  error  of  the  estimate  (SEE)  and 
Pearson  correlation  coefficient  were  also  calculated  for  both  HR  and  RR  measured  on 
each  LSDS  device  during  each  of  the  eight  activities  as  further  measurements  of 
validity.  The  SEE  is  a  measure  of  the  accuracy  of  predictions  made  with  a  regression 
line  utilizing  the  LSDS  device  measurement  of  HR  or  RR  (i.e.,  independent  variable)  to 
predict  the  criterion  measurement  of  HR  or  RR  (i.e.,  dependent  variable).  In  other 
words,  any  time  you  use  a  regression  line  to  estimate  a  subject’s  true  heart  rate  from 
the  device,  there  will  be  an  error.  This  error,  expressed  as  a  standard  deviation,  is  the 
SEE.  Just  like  the  CV,  defining  an  acceptable  SEE  depends  on  how  far  the  estimated 
measurement  can  vary  from  the  true  measurement  without  affecting  decisions.  For 
instance,  in  this  study,  a  SEE  for  HR  of  4  beatsnnin'1  would  suggest  that  the  predicted 
HR  would  vary  by  plus  or  minus  2  x  (SEE)  or  8  beats^min'1  95%  of  the  time.  Given  the 
acceptability  criteria  that  HR  should  be  within  ±  5%  of  the  gold  standard  and  the 
maximum  HR  during  running  was  -160  beats*min_1,  a  SEE  for  HR  of  4  beats^min'1 
would  be  reasonable.  Following  the  same  reasoning,  a  SEE  for  RR  of  1  breath*min  1 
would  be  reasonable.  The  percentage  of  time  a  usable  signal  (i.e.,  HR  and  RR)  was 
detected  was  determined  by  dividing  the  total  number  of  received  signals  by  the  total 
possible  number  of  signals.  The  percent  deviation  of  HR  and  RR  from  the  LSDS  device 
compared  to  the  criterion  device  was  calculated  as  device  minus  criterion  divided  by  the 
criterion.  This  measurement  was  used  for  defining  acceptability.  All  data  are  presented 
as  mean  ±  SD.  Statistical  significance  was  set  at  p<0.05. 

RESULTS 


RELIABILITY 

Figure  1  demonstrates  the  overall  change  in  HR  mean  between  trials  (i.e.  overall 
error  score)  for  each  device  combining  all  eight  activities.  None  of  the  error  scores 
were  significantly  different  from  zero.  Bland-Altman  plots  demonstrated  that  two  devices 
(CIMIT  and  VivoMetrics)  provided  more  reliable  measurements  of  HR  than  the  others 
(Figure  2).  For  convenience,  the  y-axes  are  standardized  to  highlight  the  differences  in 
reliability  between  devices.  Table  3  demonstrates  that  the  overall  prediction  intervals 
(beatsTnin"1)  for  both  CIMIT  (±15.9)  and  VivoMetrics  (±13.0)  were  similar  to  that  for  the 
criterion  measure  (±15.1)  but  were  -four  times  greater  for  Sarcos  (±54.0)  and  -six 
times  greater  for  Hidalgo  (±86.6).  In  addition,  the  overall  CV  (%)  for  CIMIT  (5.1)  and 
VivoMetrics  (4.2)  was  comparable  to  the  criterion  measure  (5.4)  but  much  greater  for 
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Sarcos  (17.5)  and  Hidalgo  (27.8)  (Table  3).  When  individual  activity  CVs  (%)  were 
examined,  CIMIT  was  comparable  to  the  criterion  measure  during  all  activities,  Hidalgo 
and  Sarcos  were  comparable  to  the  criterion  measure  only  during  low  activities,  and 
VivoMetrics  was  comparable  to  the  criterion  measure  during  all  activities  except  sit-ups 
and  push-ups.  The  ICC  reached  significance  (i.e.,  high  reliability)  in  six  of  eight  activities 
for  CIMIT,  three  of  eight  activities  for  Hidalgo,  three  of  eight  activities  for  Sarcos,  and 
three  of  eight  activities  for  VivoMetrics  (Table  3). 

Figure  3  demonstrates  the  overall  change  in  RR  mean  between  trials  (i.e.,  overall 
error  score)  for  each  LSDS  device  combining  all  eight  activities.  None  of  the  error 
scores  were  significantly  different  from  zero.  Bland-Altman  plots  demonstrated  that  two 
devices  (CIMIT  and  VivoMetrics)  provided  more  reliable  measurements  of  RR  than  the 
others  (Figure  4).  Table  4  demonstrates  that  the  overall  prediction  interval 
(breathsTnin'1)  for  CIMIT  (±5.8)  and  VivoMetrics  (±6.4)  was  similar  to  that  for  the 
criterion  measure  (±5.2)  but  was  -two  times  greater  for  Sarcos  (±9.4)  and  Hidalgo 
(±14.2).  In  addition,  the  overall  CV  (%)  for  CIMIT  (8.3)  and  VivoMetrics  (9.9)  was 
comparable  to  criterion  (6.5)  but  much  greater  for  Sarcos  (26.5)  and  Hidalgo  (41.8) 
(Table  4).  When  individual  activity  CVs  were  examined  (Table  4),  CIMIT  was 
comparable  to  the  criterion  measure  during  medium  and  high  intensity  activities  but  not 
low  intensity  activities,  Hidalgo  was  comparable  to  the  criterion  measure  only  during 
lying,  Sarcos  was  not  comparable  to  the  criterion  measure  in  any  activity  and 
VivoMetrics  was  comparable  to  the  criterion  measure  during  all  activities  except  sit-ups. 
The  ICC  reached  significance  in  seven  of  eight  activities  for  CIMIT,  four  of  eight 
activities  for  Hidalgo,  four  of  eight  activities  for  Sarcos,  and  six  of  eight  activities  for 
VivoMetrics. 

Although  cut-off  reliability  scores  were  not  defined  prior  to  the  conduct  of  this 
protocol,  a  device  can  not  be  valid  if  it  does  not  demonstrate  adequate  reliability.  Thus, 
if  a  device  failed  both  CV  and  ICC  expectations,  their  reliability  could  be  considered 
questionable.  The  devices  that  failed  these  expectations  for  HR  and  RR  and  thus 
demonstrated  questionable  reliability  are  highlighted  in  light  gray  in  Tables  3  and  4, 
respectively. 

VALIDITY 

Figure  5  demonstrates  the  device  minus  criterion  HR  mean  between  trials  (i.e., 
overall  error  score)  for  each  device  combining  all  eight  activities.  The  validity  error 
score  for  Sarcos  was  significantly  different  from  zero  indicating  that  it  significantly 
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underestimated  the  HR  compared  to  the  criterion  device.  The  remaining  devices  did  not 
significantly  over-  or  under-estimate  HR.  Bland-Altman  plots  demonstrated  that  CIMIT 
and  VivoMetrics  provided  more  valid  measurements  of  HR  than  Hidalgo  (Figure  6). 

The  overall  prediction  interval  (beatsTnin'1)  for  Hidalgo  (±86.2)  was  ~  four  to  seven 
times  greater  than  the  overall  prediction  interval  for  CIMIT  (±19.6)  and  VivoMetrics 
(±1 1 .6),  respectively.  When  individual  activity  SEEs  were  examined  (Table  5),  CIMIT 
performed  poorly  during  sit-ups,  push-ups,  and  jumping  jacks,  Hidalgo  performed  poorly 
during  all  activities,  Sarcos  performed  poorly  during  medium  and  high  intensity 
activities,  and  VivoMetrics  performed  well  during  all  activities  except  push-ups.  The 
Pearson  correlation  coefficients  were  significant  for  six  of  eight  activities  for  CIMIT,  zero 
of  eight  activities  for  Hidalgo,  four  of  eight  activities  for  Sarcos,  and  eight  of  eight 
activities  for  VivoMetrics. 

Figure  7  illustrates  the  device  minus  criterion  RR  mean  between  trials  (i.e. , 
overall  error  score)  for  each  device  combining  all  eight  activities.  The  validity  error 
score  for  Sarcos  was  significantly  different  from  zero  indicating  that  it  significantly 
underestimated  the  RR  compared  to  the  criterion.  The  remaining  devices  did  not 
significantly  over-  or  under-estimate  RR.  Bland-Altman  plots  demonstrated  similar 
prediction  intervals  (breathsnnin'1)  for  CIMIT  (±16.5),  VivoMetrics  (±21.4),  and  Hidalgo 
(±15.7)  (Figure  8).  When  individual  activity  SEEs  were  examined  (Table  6),  CIMIT, 
Hidalgo,  and  Sarcos  performed  poorly  during  medium  and  high  intensity  activities  while 
VivoMetrics  only  performed  poorly  during  sit-ups,  push-ups,  and  jumping  jacks.  The 
Pearson  correlation  coefficient  for  RR  was  significant  for  four  of  eight  activities  for 
CIMIT,  zero  of  eight  activities  for  Hidalgo,  zero  of  eight  activities  for  Sarcos,  and  five  of 
eight  activities  for  VivoMetrics. 

Although  SEE  and  Pearson  correlation  coefficient  expectations  were  not  defined 
prior  to  the  conduct  of  the  protocol,  if  a  device  failed  to  meet  fair  and  reasonable 
expectations  for  these  measures,  then  their  validity  could  be  considered  questionable. 
The  devices  that  failed  these  expectations  for  HR  and  RR  and  thus  demonstrated 
questionable  validity  are  highlighted  in  light  gray  in  Tables  5  and  6,  respectively. 

ACCEPTABILITY 

Acceptable  data  was  defined  as  HR  and  RR  within  (a)  ±  3%  of  the  criterion 
measure  90%  of  the  time  during  low  intensity  activity,  (b)  ±4%  of  the  criterion  measure 
80%  of  the  time  during  medium  intensity  activity,  and  (c)  +5%  of  the  criterion  measure 
70%  of  the  time  during  high  intensity  activity.  The  three  medium  intensity  activities  (i.e., 
sit-ups,  push-ups,  and  jumping  jacks)  were  combined  into  one  category  for  the 
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percentage  of  time  a  usable  signal  was  detected  due  to  the  short  time  of  each  of  these 
activities  (~1-2  min).  Figure  9  demonstrates  that  all  devices  met  the  percentage  of  time 
criteria  except  during  lying  and  sitting  for  Hidalgo  and  during  standing  for  CIMIT. 
Although  the  percentage  of  time  acceptability  criteria  were  generally  met,  Figure  9 
demonstrates  that  Hidalgo  had  the  lowest  percentage  of  time  a  usable  signal  was 
detected  while  VivoMetrics  had  the  highest.  CIMIT  and  Sarcos  were  in  the  middle  as 
far  as  percentage  of  time  a  usable  signal  was  received,  with  CIMIT  performing  slightly 
better  on  HR  and  Sarcos  performing  better  on  RR.  Figure  10  shows  that  VivoMetrics 
was  the  only  device  that  met  acceptability  criteria  for  HR  during  all  eight  activities. 

CIMIT  met  the  acceptability  criteria  for  HR  during  all  activities  except  push-ups.  Sarcos 
met  the  acceptability  criteria  for  HR  during  all  activities  except  sit-ups  and  push-ups  but 
there  was  a  large  percent  deviation  from  the  criterion  device  during  these  activities. 
Hidalgo  met  the  acceptability  criteria  for  HR  only  during  three  activities:  sit-ups,  jumping 
jacks,  and  running.  Figure  10  also  demonstrates  that  VivoMetrics  met  the  acceptability 
criteria  for  RR  in  six  out  of  eight  activities.  CIMIT  and  Hidalgo  met  the  RR  acceptability 
criteria  in  only  one  of  eight  activities  (i.e.,  walking).  Sarcos  failed  to  meet  the 
acceptability  criteria  for  RR  during  any  of  the  activities.  Devices  that  failed  to  meet  the 
acceptability  criteria  are  highlighted  in  dark  gray  in  Tables  5  and  6. 
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Table  3.  Reliability  of  Heart  Rate  (HR)  Measured  During  Two  Consecutive  Trials 
by  Four  Devices  (i.e.,  CIMIT,  Hidalgo,  Sarcos,  and  VivoMetrics)  and  a  Criterion 
Device  During  Eight  Activities. 


Device 


Activity 

Change  in  Mean 

Between  Trials 
(beats-min'1) 

Coefficient  of 

Variation 

(%) 

Lying 

4.712.7 

3.0 

Sitting 

4.013.3 

3.5 

Standing 

5.214.9 

4.7 

Sarcos 


HHS2SS9 


Jumping  Jacks 


Walking 

Running 


Lying 


Sitting 


Standing 


Sit-ups 


Push-ups 


Jumping  Jacks 


Walking 


Running 


Lying 


Sitting 


Standing 


Sit-ups 


Push-ups 


Jumping  Jacks 


Walking 


g 


g 


Sitting 

Standing 

Sit-ups 


Push-ups 


Jumping  Jacks 


Walking 


-5.6±8.6 


-18.6±5.6 

2.1±2.7 


0.116.3 


-0.115.4 


3.216.9 


-10.5113.3 


Limits  of 
Agreement 
(beatsTnin'1) 


15.4 


16.6 

19.8 

117.0 


.8 

.2 


111.2 


15.4 

112.6 


Intraclass 

Correlation 

Coefficient 


139.6 


13.6  159.0 


17.7  180.4 


Criterion 


2.715.0 


1.317.1 


4.414.8 


7.9+29.0 


-1.8133.6 


-14.0119.2 


-8.9124.3 


9.6114.2 


4.916.0 

2.613.6 

5.613.0 

-11.116.7 


-15.0+21.2 


-5.617.3 


-19.716.0 


4.5+16.9 


1.815.2 


1.713.8 


3.815.7 

-9.612.9 


-7.119.6 

-5.4110.9 


-6.9111.0 


4.814.7 


5.6 


8.0 


4.6 


23.7 


27.7 


9.2 


16.6 


6.2 


.0 


.2 


+9.6 


158.0 


167.2 


138.4 


+48.6 


128.4 


112.0 


17.2 

+6.0 


+10.4 


17.6 

111.4 


+5.8 


122.0 

+9.4 


0.97 

0.97 


0.97* 


0.55 


0.51 


0.26 


0.19 


0.22 


0.76* 


0.60* 


0.46 


0.31 


-0.31 


-0.38 


-0.47 


0.62* 


0.39 


0.84* 


Sitting 


Standing 


_ Sit-ups _ 

Push-ups 


Jumping  Jacks 
Walking 


Running 


^<0.05,  white  indicates  acceptable  reliability;  light  gray  indicates  questionable  reliability 
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Table  4.  Reliability  of  Respiration  Rate  (RR)  Measured  During  Two  Consecutive 
Trials  by  Four  Devices  (i.e.,  CIMIT,  Hidalgo,  Sarcos,  and  VivoMetrics)  and  a 


Criterion  Device  During  Eight  Activities. 


Device 

Activity 

Change  in  Mean 

Between  Trials 
(breaths-min'1) 

Coefficient  of 

Variation 

(%) 

Limits  of 

Agreement 

(breathsnnin"1) 

Intraclass 

Correlation 

Coefficient 

CIMIT 

Lying 

-1.2±2.3 

14.7 

±4.6 

Sitting 

-0.5±  1 .5 

7.3 

±3.0 

0.75* 

Standing 

0.3±2.9 

15.7 

±5.8 

0.86* 

Sit-ups 

-2.1±4.9 

13.3 

±9.8 

0.54 

Push-ups 

0.1±2.9 

7.5 

±5.8 

0.73* 

Jumping  Jacks 

-1.5+5. 5 

9.5 

±11.0 

0.70* 

Walking 

-4.3±  1 .8 

4.9 

±3.6 

0.64* 

Running 

0.5±2.9 

5.1 

±5.8 

| 

Hidalgo 

Lying 

-0.8±  1 .4 

8.8 

±2.8 

■| 

Sitting 

-0.7±2.3 

13.7 

±4.6 

0.90* 

Standing 

-0.3±2.0 

9.9 

±4.0 

0.98* 

12.8 

±7.6 

0.43 

Push-ups 

-2.6±4.1 

13.3 

±8.2 

0.77* 

'T  ,  ... 

Jumping  Jacks 

-4.1  ±5.9 

18.3 

wm 

iSBMH 

-0.02 

Walking 

4.6±7.5 

±15.0 

0.36 

Running 

-1.5±9.8 

mmmam 

±19.6 

-0.31 

Sarcos 

-1.4±2.4 

85.7 

eamifi 

Sitting 

0.2±1.6 

117.0 

B3EMWI 

-'Szjsi 

Standing 

-0.3±  1 .3 

55.6 

±2.6 

0.79* 

Sit-ups 

-0.4±2.5 

13.9 

±5.0 

0.75* 

Push-ups 

-0.513.7 

18.3 

±7.4 

0.83* 

Jumping  Jacks 

-0.5±5.9 

36.2 

±11.8 

0.50 

Walking 

-2.0110.7 

93.3 

±21.4 

0.20 

Running 

3.717.1 

39.8 

±14.2 

0.65* 

VivoMetrics 

Lying 

10.2 

±4.8 

0.47 

Sitting 

-0.211.2 

5.1 

±2.4 

0.80* 

Standing 

1.112.3 

9.3 

±4.6 

0.74* 

Sit-ups 

0.918.6 

17.6 

±17.2 

mmm 

Push-ups 

-2.714.5 

8.4 

±9.0 

0.87* 

Jumping  Jacks 

-1.812.1 

2.9 

±4.2 

0.98* 

Walking 

-4.911.3 

3.5 

±2.6 

-0.27 

-0.915.1 

8.5 

±10.2 

0.81* 

Criterion 

Lying 

-1.311.3 

5.6 

±2.6 

0.78* 

-0.611.2 

5.6 

±2.4 

0.74* 

Standing 

-1.212.3 

9.1 

±4.6 

0.79* 

Sit-ups 

-1.411.3 

5.5 

±2.6 

0.93* 

Push-ups 

-0.712.6 

6.0 

±5.2 

0.70* 

Jumping  Jacks 

-0.012.6 

4.8 

±5.2 

0.93* 

Walking 

-3.612.6 

7.2 

±5.2 

0.81* 

Running 

-1.313.9 

6.2 

±7.8 

0.92* 

*P<0.05,  white  indicates  acceptable  reliability;  light  gray  indicates  questionable  reliability 
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Table  5.  Validity  of  Heart  Rate  (HR)  Measured  During  Two  Consecutive  Trials  by 
Four  Devices  (i.e.,  CIMIT,  Hidalgo,  Sarcos,  and  VivoMetrics)  During  Eight 
Activities. 


g 


Sitting 


Standing 


Sit-ups 


Push-ups 


Jumping  Jacks 


Walking 


Running 


*P<0.05,  white  indicates  acceptable  validity;  light  gray  indicates 
dark  gray  indicates  unacceptable  validity 


questionable  validity, 
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Table  6.  Validity  of  Respiration  Rate  (RR)  Measured  During  Two  Consecutive 
Trials  by  Four  Devices  (i.e.,  CIMIT,  Hidalgo,  Sarcos,  and  VivoMetrics)  During  Eight 
Activities. 


Device 


Activity 


Sit-ups 


Push-ups 


VivoMetrics 


Device  -  Criterion 
(breathsnnin'1) 


-4.9  ±  7.1 


-6.2  ±  8.2 


Standard 
Error  of  the 
Estimate 


0.9 


Jumping  Jacks 

2.5  ±  5.0 

Walking 

0.1  ±3.9 

Running 

Lying 

-6.1  ±  7.0 

Sitting 

111. ttif 0 I!??  it :-t  t|; :  ?! f.|! I ?|> :!?! ?! mM. ?? 

Standing 

-2.5  ±8.8 

Sit-ups 

Push-ups 

-8.4  ±4.7 

Jumping  Jacks 

-13.9  ±8.7 

Walking 

-0.9  ±  7.2 

Running 

-19.5  ±7.2 

Lying 

-14.7  ±2.4 

Sitting 

-14.9  ±  1.6 

Standing 

-16.3  ±2.7 

Sit-ups 

-15.7  ±8.6 

Push-ups 

-14.2  ±9.7 

Jumping  Jacks 

-23.4  ±  11.7 

Walking 

-17.7  ±7.6 

Running 

-30.9  ±  8.7 

Lying 

-0.1  ±0.7 

Sitting 

0.2  ±0.8 

Standing 

0.1  ±  1.3 

Sit-ups 

-0.5  ±  10.0 

Push-ups 

4.1  ±9.1 

Jumping  Jacks 

12.9  ±  13.7 

II 

g 

0.8  ±  0.9 

g 

0.4  ±0.3 

Limits  of  Pearson 

Agreement  Correlation 

(breaths'min'1)  Coefficient 


*P<0.05,  white  indicates  acceptable  validity;  light  gray  indicates 
dark  gray  indicates  unacceptable  validity 


0.30 


0.86* 


0.99* 


questionable  validity, 
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Trial  1 -Trial  2  Trial  1  -  Trial  2 

(beats-min"1)  (beats-min"1) 


Figure  2.  Bland-Altman  Plots  for  Individual  Heart  Rate  Reliability  Error  Scores 

Combining  Eight  Activities 
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Figure  3.  Respiration  Rate  Reliability  Error  Scores  Combining  Eight  Activities 
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Figure  4.  Bland-AItman  Plots  for  Individual  Respiration  Rate  Reliability  Error 

Scores  Combining  Eight  Activities 
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Figure  5.  Heart  Rate  Validity  Error  Scores  Combining  Eight  Activities 

Heart  Rate 
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Device 


Device  -  Criterion  Device  -  Criterion 
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Figure  6.  Bland-Altman  Plots  for  Individual  Heart  Rate  Validity  Error  Scores 

Combining  Eight  Activities 
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Device  -  Criterion  Device  -  Criterion 

(breaths-min'1)  (breaths-min1) 


Figure  8.  Bland-Altman  Plots  for  Individual  Validity  Respiration  Rate  Error 

Scores  Combining  Eight  Activities 
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Figure  9.  Percentage  of  Time  Usable  Signal  Detected 
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Figure  10.  Percent  Deviation  from  Criterion  Device 
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DISCUSSION 


The  purpose  of  this  study  was  to  evaluate  the  reliability  and  validity  of  four  LSDS 
devices  (CIMIT,  Hidalgo,  Sarcos,  and  VivoMetrics)  for  measuring  heart  rate  (HR)  and 
respiration  rate  (RR)  against  criterion  devices  (i.e.,  3-lead  ECG  and  metabolic  cart, 
respectively).  This  was  done  by  comparing  the  HR  and  RR  obtained  by  these  four 
LSDS  devices  against  the  criterion  devices  during  eight  activities  (i.e.,  lying,  sitting, 
standing,  sit-ups,  push-ups,  jumping  jacks,  walking,  and  running)  on  two  occasions. 

The  results  demonstrated  that  all  of  the  devices  demonstrated  adequate 
reliability  by  hypothesis  testing  since  none  of  the  error  scores  for  HR  and  RR  between 
trials  (Trial  1  -  Trial  2)  was  significantly  different  from  zero.  However,  the  difficulty  with 
hypothesis  testing  is  that  with  large  standard  deviations  and  low  subject  numbers,  the 
result  is  biased  to  detect  no  differences  between  methods  (1 ). 

Recent  studies  measuring  reliability  and  validity  of  biomedical  equipment  have 
employed  Bland-Altman  95%  limits  of  agreement  (LOA)  plots  in  order  to  more  closely 
examine  individual  error  scores  for  biomedical  devices  (7,16,17,20).  Devices  that 
demonstrate  individual  repeatability  error  scores  that  have  a  tight  95%  prediction 
interval  around  zero  are  more  reliable  devices  (4,14).  Utilizing  Bland-Altman  criteria, 
Figure  2  clearly  demonstrates  that  CIMIT  and  VivoMetrics  were  more  reliable  for  HR 
measurements  than  Hidalgo  and  Sarcos  given  their  tight  95%  prediction  interval  around 
zero.  Repeated  measurements  of  HR  by  Hidalgo  and  Sarcos  could  vary  by  as  much  as 
±87  beatS'min'1  and  ±54  beatSTnin'1,  respectively,  of  the  mean  difference  in  HR 
between  trials.  This  prediction  interval  is  too  large  for  practical  purposes.  The  CIMIT, 
VivoMetrics,  and  criterion  devices  all  had  similar  prediction  intervals  of  ±15  beatSTnin’1 
of  the  mean  difference  between  trials.  Figure  4  also  demonstrates  that  CIMIT, 
VivoMetrics,  and  Sarcos  were  more  reliable  for  RR  measurements  than  the  Hidalgo 
device  for  similar  reasons. 

The  CV  and  the  ICC  have  also  been  recommended  as  useful  tools  for  analyzing 
repeatability  of  biomedical  instrumentation  (13).  The  CV,  which  represents  the 
variability  between  repeated  measurements,  has  been  previously  reported  for  ECG 
machines  (19,23),  and  is  similar  to  the  5.4%  reported  for  the  criterion  device  in  this 
study.  Given  the  prototype  development  stage  of  these  devices  as  a  LSDS,  accepting 
a  CV  that  is  double  (i.e.,  10%)  the  standard  CV  for  measurement  of  HR  would  be 
reasonable.  In  this  scenario,  only  CIMIT  and  VivoMetrics  met  this  expectation  during  all 
eight  activities.  Hidalgo  did  not  meet  this  CV  expectation  for  HR  measurements  during 
push-ups,  jumping  jacks,  walking,  or  running,  while  Sarcos  did  not  meet  this 
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expectation  during  sit-ups,  push-ups,  or  walking  (Table  3).  The  CV  for  measuring  RR  by 
the  metabolic  cart  has  also  been  previously  reported  and  is  similar  to  the  6.5%  reported 
in  this  study  (14,15).  Again,  given  the  development  stage  of  these  devices,  doubling 
the  CV  (i.e.,  -13.0%)  for  measuring  RR  between  trials  would  be  reasonable.  In  this 
scenario,  the  CV  expectation  was  met  by  VivoMetrics  in  seven  of  eight  activities,  by 
CIMIT  in  four  of  eight  activities,  by  Hidalgo  in  three  of  eight  activities,  and  by  Sarcos  in 
zero  of  eight  activities  (Table  4). 

The  ICC  has  also  been  widely  used  as  a  measure  of  repeatability  between  trials 
versus  the  Pearson  correlation  coefficient  because  systematic  errors  in  measurement 
are  reflected  by  a  reduction  in  the  value  of  the  coefficient  (8,12).  Utilizing  this  technique 
for  assessing  reliability,  none  of  the  devices  achieved  a  significant  ICC  during  all  eight 
activities  for  either  HR  or  RR.  CIMIT  demonstrated  the  greatest  number  of  significant 
correlation  coefficients  for  both  HR  and  RR. 

Considering  all  four  methods  of  measuring  reliability,  it  is  clear  that  both  CIMIT 
and  VivoMetrics  were  more  reliable  than  Sarcos  and  Hidalgo.  Based  on  the  weight  of 
HR  reliability  evidence,  the  devices  would  demonstrate  the  following  rank  order:  (a) 
CIMIT,  (b)  VivoMetrics,  (c)  Sarcos,  and  (d)  Hidalgo.  Based  on  the  weight  of  RR 
reliability  evidence,  the  devices  would  demonstrate  the  following  rank  order:  (a) 
VivoMetrics,  (b)  CIMIT,  (c)  Hidalgo,  and  (d)  Sarcos. 

Hypothesis  testing  of  the  mean  difference  in  HR  and  RR  measured  by  the  LSDS 
device  and  criterion  device  demonstrated  that  Sarcos  was  the  only  device  in  which  HR 
and  RR  validity  error  scores  were  significantly  different  from  zero.  All  of  the  other 
devices  demonstrated  acceptable  validity  error  scores. 

However,  Bland-Altman  95%  LOA  plots  for  individual  validity  HR  error  scores 
demonstrate  that  VivoMetrics  provided  more  valid  measures  of  HR  than  CIMIT, 

Hidalgo,  or  Sarcos.  The  prediction  intervals  for  CIMIT  and  Sarcos  were  double  that  for 
Vivometrics  while  the  prediction  interval  for  Hidalgo  was  approximately  eight  times 
wider  than  the  prediction  interval  for  VivoMetrics.  The  Bland-Altman  plots  for  individual 
RR  validity  error  scores  demonstrate  that  CIMIT,  Hidalgo,  and  Vivometrics  performed 
similarly  while  the  prediction  interval  for  Sarcos  was  30%  wider  than  for  the  others. 

Given  that  expectations  for  the  SEE  and  Pearson  correlation  coefficient  were  not 
defined  prior  to  the  conduct  of  the  protocol,  a  device  should  not  be  considered 
unacceptable  based  on  these  criteria.  Furthermore,  the  Pearson  correlation  coefficient 
does  not  necessarily  measure  agreement  between  two  device  but  rather  association 
between  them  (12).  Nonetheless,  if  a  device  failed  to  meet  both  of  these  criteria,  then 
the  validity  of  that  device  should  be  considered  questionable. 
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The  results  in  total  of  the  different  methods  of  assessing  validity  also  clearly 
demonstrate  that  CIMIT  and  VivoMetrics  are  more  valid  devices  for  measuring  HR  and 
RR  than  Hidalgo  and  Sarcos.  Based  on  the  weight  of  HR  validity  evidence,  the  devices 
would  demonstrate  the  following  rank  order:  (a)  VivoMetrics,  (b)  CIMIT,  (c)  Sarcos,  and 
(d)  Hidalgo.  Based  on  the  weight  of  RR  validity  evidence,  the  devices  would 
demonstrate  the  following  rank  order:  (a)  VivoMetrics,  (b)  CIMIT,  (c)  Hidalgo,  and  (d) 
Sarcos. 

The  acceptability  criteria  that  were  defined  prior  to  the  conduct  of  the  protocol 
stated  that  devices  would  demonstrate  a  HR  and  RR  within  (a)  ±  3%  of  the  criterion 
measure  90%  of  the  time  during  low  intensity  activity,  (b)  ±4%  of  the  criterion  measure 
80%  of  the  time  during  medium  intensity  activity,  and  (c)  ±5%  of  the  criterion  measure 
70%  of  the  time  during  high  intensity  activity.  Although  most  devices  met  the 
percentage  of  time  a  usable  signal  was  received  criteria  during  most  activities  except 
for  Hidalgo  during  lying  and  sitting  and  CIMIT  during  standing,  Figure  9  clearly  shows 
that  the  Hidalgo  device  demonstrated  decreased  performance  in  this  area.  The  reason 
for  this  is  that  the  Hidalgo  system  was  an  adhesive-based  system  that  frequently  fell  off 
subjects  during  high  intensity  activity  and  failed  to  adhere  properly  during  low  intensity 
activity.  The  reasons  for  the  lower  percentage  of  time  a  usable  signal  was  received  for 
HR  from  the  Sarcos  system  was  that  the  belt  frequently  fell  down  from  the  expected 
chest  position  during  high  activity.  Reasons  for  the  decreased  percentage  of  time  a 
signal  was  received  for  RR  in  the  CIMIT  system  is  not  entirely  known  but  may  have 
been  due  to  signal  transduction  from  the  belt  to  the  receiver.  Reasons  for  the  high 
percentage  of  time  a  signal  was  received  for  both  HR  and  RR  for  the  VivoMetrics 
system  was  most  likely  due  to  the  constraining  vest  type  design.  Although  this  user 
design  may  be  appropriate  for  collecting  reliable  and  valid  data,  it  was  not  an 
acceptable  user  design  for  the  Soldier  in  the  field  (4). 

Figure  10  shows  that  VivoMetrics  was  the  only  device  that  met  acceptability 
criteria  for  HR  during  all  eight  activities.  CIMIT  met  acceptability  criteria  in  6  of  8 
activities,  Hidalgo  in  3  of  8  activities,  and  Sarcos  in  6  of  8  activities.  None  of  the 
devices  met  the  acceptability  criteria  for  RR  during  all  activities.  However,  VivoMetrics 
met  the  acceptability  criteria  for  RR  during  five  of  the  eight  activities  whereas  CIMIT  and 
Hidalgo  met  the  acceptability  criteria  in  only  one  of  eight  activities.  Sarcos  failed  to 
meet  the  acceptability  criteria  for  RR  during  any  of  the  activities.  The  results  of  this 
study  are  not  surprising  given  that  VivoMetrics  is  an  FDA-certified  device.  Thus,  certain 
reliability  and  validity  criteria  were  already  met  before  FDA-certification. 
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CONCLUSIONS 


Based  on  the  reliability  and  validity  results  of  this  study,  VivoMetrics  was  the 
most  reliable  and  valid  device  based  on  several  statistical  measures.  CIMIT  was  the 
next  most  reliable  and  valid  device  and  provided  more  valid  measurements  of  HR  than 
RR.  The  Sarcos  and  Hidalgo  devices  should  not  be  evaluated  further  in  their  current 
configuration  based  on  the  poor  reliability  and  validity  of  HR  and  RR  measurements  in 
this  study.  Even  though  the  VivoMetrics  device  was  the  most  reliable  and  valid  device, 
the  form  factor  was  not  acceptable  to  the  Soldier  in  the  field.  Thus,  a  future  LSDS 
system  may  need  to  utilize  components  of  each  system  to  meet  all  the  needs  of  the 
Soldier.  Future  research  should  also  examine  other  physiological  variables  related  to 
dead/alive  status  such  as  body  position,  movement,  blood  pressure,  and  body 
temperature  to  more  accurately  determine  life  sign  status. 
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